The present invention relates to methods and apparatus for displaying images, and more particularly, to display methods and apparatus which utilize multiple displaced portions of an output device, e.g., liquid crystal display, to represent a single pixel of an image.
Color display devices have become the principal display devices of choice for most computer users. The display of color on a monitor is normally achieved by operating the display device to emit light, e.g., a combination of red, green, and blue light, which results in one or more colors being perceived by the human eye.
In cathode ray tube (CRT) display devices, the different colors of light are generated via the use of phosphor coatings which may be applied as dots in a sequence on the screen of the CRT. A different phosphor coating is normally used to generate each of the three colors, red, green, and blue. The coating results in repeating sequences of phosphor dots which, when excited by a beam of electrons, will generate the colors red, green and blue.
The term pixel is commonly used to refer to one spot in, e.g., a rectangular grid of thousands of such spots. The spots are individually used by a computer to form an image on the display device. For a color CRT, where a single triad of red, green and blue phosphor dots cannot be addressed, the smallest possible pixel size will depend on the focus, alignment and bandwidth of the electron guns used to excite the phosphors. The light emitted from one or more triads of red, green and blue phosphor dots, in various arrangements known for CRT displays, tend to blend together giving, at a distance, the appearance of a single colored light source.
In color displays, the intensity of the light emitted corresponding to the additive primary colors, red, green, and blue, can be varied to get the appearance of almost any desired color pixel. Adding no color, i.e., emitting no light, produces a black pixel. Adding 100 percent of all three colors results in a white pixel.
Portable computing devices, including hand held devices and portable computers, tend to use liquid crystal displays (LCD) or other flat panel display devices 102, as opposed to CRT displays. This is because flat panel displays tend to be small and light weight as compared to CRT displays. In addition, flat panel displays tend to consume less power than comparable sized CRT displays making them well suited for battery powered applications.
Color LCD displays are exemplary of display devices which utilize multiple distinctly addressable elements, referred to herein as pixel sub-components or pixel sub-elements, to represent each pixel of an image being displayed. Normally, each pixel element of a color LCD display comprises three non-square elements, i.e., red, green and blue (RGB) pixel sub-components. Thus, a set of RGB pixel sub-components together make up a single pixel element. Known LCD displays generally comprise a series of RGB pixel sub-components which are commonly arranged to form stripes along the display. The RGB stripes normally run the entire length of the display in one direction. The resulting RGB stripes are sometimes referred to as xe2x80x9cRGB stripingxe2x80x9d. Common LCD monitors used for computer applications, which are wider than they are tall, tend to have RGB stripes running in the vertical direction.
FIG. 1 illustrates a known LCD screen 200 comprising a plurality of rows (R1-R12) and columns (C1-C16). Each row/column intersection forms a square which represents one pixel element. FIG. 2 illustrates the upper left hand portion of the known display 200 in greater detail.
Note in FIG. 2 how each pixel element, e.g., the (R1, C4) pixel element, comprises three distinct sub-element or sub-components, a red sub-component 206, a green sub-component 207 and a blue sub-component 208. Each known pixel sub-component 206, 207, 208 is ⅓ or approximately ⅓ the width of a pixel while being equal, or approximately equal, in height to the height of a pixel. Thus, when combined, the three ⅓ width pixel sub-components 206, 207, 208 form a single pixel element.
As illustrated in FIG. 1, one known arrangement of RGB pixel sub-components form what appear to be vertical color stripes down the display 200. Accordingly, the arrangement of ⅓ width color sub-components 206, 207, 208, in the known manner illustrated in FIGS. 1 and 2, is sometimes called xe2x80x9cvertical stripingxe2x80x9d.
Traditionally, each set of pixel sub-components for a pixel element is treated as a single pixel unit. Accordingly, in known systems luminous intensity values for all the pixel sub-components of a pixel element are generated from the same portion of an image. Consider for example, the image represented by the grid 220 illustrated in FIG. 3. In FIG. 3 each square represents an area of an image which is to be represented by a single pixel element, e.g., a red, green and blue pixel sub-component of the corresponding square of the grid 230. In FIG. 3 a shaded circle is used to represent a single image sample from which luminous intensity values are generated. Note how a single sample 222 of the image 220 is used in known systems to generate the luminous intensity values for each of the red, green, and blue pixel sub-components 232, 233, 234. Thus, in known systems, the RGB pixel sub-components are generally used as a group to generate a single colored pixel corresponding to a single sample of the image to be represented.
While individual RGB sub-component intensities can be varied to support a wide range of different colors, frequently, only a limited number of colors are actually supported by the display hardware and/or software. A supported set of colors is referred to commonly as a color palette. Each color in a color palette corresponds to a different combination of R, G, and B pixel sub-component luminance intensity values. While a computer system may support multiple color palettes, normally only one palette may be used to generate the images displayed at any given time.
In the case of text rendering, a user normally selects a foreground and background color to be used. Commonly, a dark foreground and a light background color is selected to produce dark text on a light colored field. Such an approach mimics printed text which frequently tends to be black on a white background.
As an alternative to a dark foreground and a light background, a user may select a light foreground and a dark background. While such a text option is less common, it is sometimes used, e.g., to highlight text on the screen.
In the case where text is rendered at pixel resolution as is commonly done, pixels used to represent a character are set to the selected foreground color, e.g., black, and pixels used to represent the background are set to white. As discussed above, to produce a black pixel the R, G, and B pixel sub-components of the black pixel are set to output the minimum possible luminous intensity. In the case of a white pixel the R, G, and B pixel sub-components are set to their maximum luminous intensity.
Frequently, because of the relatively low display resolution of most video display devices, not enough pixels are available to draw smooth character shapes, especially at common text sizes of 10, 12, and 14 point type.
Portable computing devices, and hand held computing devices in particular, face power consumption problems and, in many cases, cost constraints, which are frequently less of an issue in desk top computing devices.
As discussed above, power constraints often lead to the use of LCD display devices in an attempt to minimize power consumption. Power concerns also often result in the use of a processor, e.g., CPU, designed with power saving features. Since there are more desk top PCs than portable computers, most CPU manufactures give priority to developing fast CPUs for desktop computers, as opposed to CPUs with power saving features for portable computers. For this reason processors in portable computing devices tend to lag behind their desktop counterparts in terms of processing power.
In the case of hand held computing devices, e.g., personal assistant devices, cost tends to be a limiting factor in terms of both processing power and the amount of memory included in the device.
Thus, while portable and hand held computing devices face many of the same image rendering problems encountered in other image rendering environments, such as desk top personal computers, frequently less processing power and memory is available in the portable computer and hand held environments to address the image rendering problems.
In view of the above discussion, it is apparent that there is a need for improved methods of rendering and displaying images including text. It is desirable that at least some new methods and apparatus support the rendering of text in portable computers and hand held computing devices, where processing and memory resources are limited.
The present invention is directed to methods and apparatus for displaying images utilizing multiple distinct portions of an output device, e.g., an LCD display, to represent a single pixel of an image.
In accordance with the present invention, each pixel sub-component of a pixel element is treated as an independent luminance source. This provides for an increase in resolution of up to 3 times in the direction perpendicular to the direction of RGB striping in the case of striped displays.
The rendering of images, e.g., text characters, in accordance with the present invention involves several steps. These include a rasterization step, an optional glyph caching step, and a device, e.g., display, driver processing step. The rasterization step may include scaling, hinting and scan conversion steps. The rasterization step produces a glyph which includes a monochrome bitmap representation of an image, e.g., a character.
The optional caching step of the present invention involves the storage of glyphs produced by the rasterization operation. When a glyph is generated by the rasterization operation, it is placed in the glyph cache for possible future use. If a character corresponding to a previously rasterized and cached glyph is to be output, the glyph may be obtained from the cache thereby avoiding the need to perform the rasterization operation.
In accordance with one embodiment of the present invention, separate R, G, and B luminous intensity level values associated with each pixel element are packed, e.g., compressed, into a single value referred to herein as a packed pixel value. In this manner, the amount of memory required to store a glyph can be significantly reduced. In one exemplary embodiment, a single 8 bit packed pixel value is used to represent the R, G, B pixel sub-component intensity values associated with each pixel element of an image. This represents a significant savings in memory over systems which use 8 bits to represent each of the R, G, B pixel sub-component luminous intensity values throughout the rendering process.
Prior to display, glyphs from the rasterization step or the glyph cache are processed to generate corresponding pixel values in a format that is supported by a display adapter and/or display device to which the pixel values will be supplied. A pixel value is a value that is used to control the luminous intensity of a pixel element. Accordingly, a pixel value may be an R, G, or B luminance intensity value or, in the case of a palletized display system, a number that is used to index into a table, e.g., color palette, that includes the actual R, G, B luminous intensity values used by the display device. In a palletized display embodiment, the display device or display adapter converts color palette values associated with individual pixels into actual R, G, and B luminous intensity values that are then used to control the R, G, and B luminous intensity of pixel sub-components.
The rasterization operation includes a scan conversion operation and, in various embodiments, optional scaling and hinting operations. As part of the scaling operation a high resolution representation of an image, e.g., character to be displayed, is super sampled in at least the dimension perpendicular to the direction of RGB striping. After hinting of the scaled image, a scan conversion operation is performed. In accordance with one feature of the present invention, the scan conversion operation is a weighted scan conversion operation wherein the hinted image is divided into equal sized portions, e.g., segments and different numbers of segments are used for determining the R, G, and B luminous intensity values.
One particular exemplary embodiment, uses super sampling by a factor of 16, resulting in a super sampled image having 16 segments corresponding to each pixel element. In the particular exemplary embodiment, the implemented weighted scan conversion operation uses 5 image segments to determine the red pixel sub-component luminous intensity value, 9 image segments to determine the green luminous intensity value and 2 image segments to determine the blue luminous intensity value.
In the case of a weighted scan conversion operation, the number N of possible R, G, and B luminous intensity value combinations for a pixel element is equal to the number of possible R values, times the number of possible G values times the number of possible B values. In the 5, 9, 2 scaling case, N corresponds to a total of (6xc3x9710xc3x973) 180 possible values.
In accordance with the present invention, rather than representing the R, G and B luminous intensity values resulting from scan conversion operation as three separate values, e.g., an 8 bit R value, an 8 bit G value, and an 8 bit B value, the R, G, B luminous intensity values corresponding to a pixel element are specified as a single number, referred to herein as a packed pixel value. The packed pixel value indicates which one of the possible sets of R, G, and B luminous intensity values a pixel element corresponds. In the 5, 9, 2, case, an 8 bit packed pixel value can be used to easily represent the 180 different possible R, G, B luminous intensity values.
The packed pixel values generated in accordance with the present invention allow a character glyph to be represented using far fewer bits than would be necessary if separate R, G, and B luminous intensity values were stored. The reduction in the amount of data used to represent a glyph using the packing technique of the present invention reduces the amount of bandwidth required to transmit a glyph over a bus as compared to systems which do not pack the luminous intensity values. The small size glyph of the present invention also has the advantage of allowing more glyphs to be stored in the same amount of memory than would be possible if packed pixel values were not used to represent the glyph. Both of these advantages can be important particularly in the case of hand held and portable computing devices where memory and bus bandwidth may be relatively limited.
When a glyph is to be displayed, the set of packed pixel values representing the glyph is supplied to the display driver. The display driver is responsible for performing one or more image processing operations to the data representing the image, e.g., character, to be displayed. The display driver converts the packed pixel values used to represent the glyph into pixel values, e.g., separate 8 bit R, G and B luminance intensity values or color palette values, having a format that is used by the display adapter or display device to which the image data is being output. The display driver may also perform gamma correction operations, color compensation operations and/or color palette selection operations which involve the selection of colors from a limited color palette that is in use at any given time.
In some embodiments the processing operations performed by the display driver are implemented as sequential processing operations. However, in other embodiments of the present invention, the processing operations performed by the display driver are combined into a simple look-up table operation.
For this purpose one or more look-up tables are provided as part of the display driver. Each look-up table corresponds to a different set of device driver operations or input values used to transform packed pixel values into pixel values having a format used by the display adapter or display device. For example, one look-up table may be provided for each of a plurality of supported display devices having different gamma values or pixel value format requirements. In addition, separate look-up tables are used for each supported color palette. For example, one look-up table generated using a color palette associated with Internet applications, is used to process packed pixel values when the Internet application color palette is the display device""s active palette. A different look-up table is used when the display device and/or display adapter has a different active color palette.
Each look-up table includes one entry for each potential packed pixel value. The entries are arranged in the table so that the Nth entry corresponds to the Nth packed pixel value in a set of potential packed pixel values. In the case of a palletized display system, each table entry is normally a single pixel value, i.e., a color palette value. In non-palletized systems, each entry is normally a set of R, G, and B pixel sub-component luminous intensity values in a format that can be used by a display adapter and/or display device. In the exemplary case of 180 possible packed pixel values, a look-up table would include 180 entries, each entry corresponding to a different one of the 180 packed pixel values. In one embodiment, each table entry is pre-computed by performing the implemented display driver processing operations using the packed pixel value, to which the output pixel value corresponds, as input.
By using the packed pixel values included in a glyph as indices into the display device look-up table of the present invention, a set of output pixel values in a form utilized by the attached display adapter or display device is obtained. Since the look-up table values are pre-computed, i.e., computed prior to use in the display driver, the need to perform gamma correction, color compensation and/or palette selection operations in real time during image rendering is avoided.
Various features of the present invention are directed to the creation of device driver look-up tables that can be used in the above described manner as will be discussed in detail below.
The present invention allows foreground and background colors to be specified for use in rendering images such as text. When pixel value look-up tables are used to implement the display driver, in accordance with the present invention, a look-up table is normally provided to support a pair of colors which may be used in combination as foreground and background colors. In accordance with the present invention, which color is used as the foreground color and which color is used as the background color, is determined by the manner in which the pixel value look-up table is accessed. In one such embodiment foreground and background colors are interchanged by simply reversing the order in which the pixel value look-up table, corresponding to the utilized color pair, is accessed.
Numerous additional features, embodiments, and advantages of the methods and apparatus of the present invention are set forth in the detailed description which follows.